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Introduction 



About TIMSS 



IEA’s Trends in International Mathematics and Science Study (TIMSS) 
provides valuable information about students’ mathematics and 
science achievement in an international context. TIMSS assesses stu- 
dents at the fourth and eighth grades, and also collects a wealth of data 
from their schools and teachers about curriculum and instruction in 
mathematics and science. TIMSS findings have been used by many 
countries around the world in their efforts to improve teaching and 
learning mathematics and science in their schools. 

Involving more than 60 countries, TIMSS 2007 is the most recent 
in the four-year cycle of studies to measure trends in students’ math- 
ematics and science achievement. The first TIMSS assessments were 
in 1995 in 41 countries, the second in 1999 involved 38 countries, and 
TIMSS 2003 involved more than 50 countries. The majority of coun- 
tries participating in TIMSS 2007 will have trend data, some covering 
more than a decade back to 1995. 

TIMSS Advanced 2008 

First administered in 16 countries in 1995, TIMSS Advanced assesses 
school-leaving students with special preparation in advanced math- 
ematics and physics. Since the 1995 assessment, however, TIMSS has 
not assessed students at the end of secondary school. Recognizing 
the strong link between scientific competence and economic pro- 
ductivity, and given the relatively long time period since the 1995 
assessments, countries around the world have expressed interest in 
participating in TIMSS Advanced. They want internationally compara- 
tive data about the achievement of their students enrolled in advanced 
courses designed to lead into science-oriented programs in university. 
By joining TIMSS Advanced, countries that participated in 1995 can 
determine whether the achievement of students having taken advanced 
coursework has changed over time. Countries participating in TIMSS 
Advanced for the first time can assess the comparative standing in 
mathematics and physics in an international context. 
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Introduction 




TIMSS uses the curriculum, broadly defined, as the major orga- 
nizing concept in considering how educational opportunities are 
provided to students, and the factors that influence how students use 
these opportunities. To begin the process of defining the topics to be 
assessed in TIMSS Advanced, this document built on the composition 
of the 1995 Advanced Mathematics and Physics assessments to draft 
a Framework for TIMSS Advanced 2008. The description of cogni- 
tive domains also benefited from the TIMSS developmental project, 
funded by a number of countries, to enable reporting TIMSS 2007 
results according to cognitive domains. The first draft of this docu- 
ment was thoroughly reviewed by participating countries, and updated 
accordingly. Countries provided feedback about the topics included 
in their curricula in advanced mathematics and physics courses, and 
made recommendations about the desirability and suitability of assess- 
ing particular topics. 

Developing and Implementing TIMSS Advanced 2008 

TIMSS, including TIMSS Advanced, is a major undertaking of the 
IEA. IEA has entrusted responsibility for the overall direction and 
management of the project to its TIMSS & PIRLS International Study 
Center at Boston College. In carrying out TIMSS, the TIMSS & PIRLS 
International Study Center works closely with the IEA Secretariat in 
Amsterdam on country membership and translation verification, the 
IEA Data Processing Center in Hamburg on database creation and 
documentation, Statistics Canada in Ottawa on sampling, and Educa- 
tional Testing Service in Princeton, New Jersey on the psychometric 
scaling of the data. IEA is extremely grateful to Chinese Taipei and 
Norway, in particular, for providing support for TIMSS Advanced. 
The TIMSS Norwegian Team is playing a significant role in developing 
items for TIMSS Advanced. As in 1995, Svein Lie from Norway is the 
Physics Coordinator. Also, as in 1995, Bob Garden from New Zealand 
is the Advanced Mathematics Coordinator. 
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MATHEMATICS 
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Overview 



The advanced mathematics assessment framework for TIMSS 
Advanced 2008 is organized around two dimensions: a content dimen- 
sion specifying the domains or subject matter to be assessed within 
mathematics (i.e., algebra, calculus, and geometry) and a cognitive 
dimension specifying the domains or thinking processes to be assessed 
(i.e., knowing, applying, and reasoning). The cognitive domains 
describe the sets of behaviors expected of students as they engage with 
the mathematics content. Exhibit 1 shows the target percentages of 
testing time devoted to each content and cognitive domain for the 
advanced mathematics assessment. 



Exhibit 1 : Target Percentages for the TIMSS Advanced Mathematics 
Assessment Devoted to Content and Cognitive Domains 



Content Domains 


Percentages 


Algebra 


35% 


Calculus 


35% 


Geometry 


30% 


Cognitive Domains 


Percentages 


Knowing 


35% 


Applying 


35% 


Reasoning 


30% 
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Advanced Mathematics 
Content Domains 



Essentially, the TIMSS 2008 Advanced Mathematics assessment consists 
of the traditional areas of algebra, functions, calculus, geometry, and 
trigonometry The first category, algebra, includes much of the algebra 
and functions content that provides the foundation for mathematics at 
the college or university level. Topics from these areas occupy a sub- 
stantial amount of the time devoted to pre-university mathematics. 

Since calculus is a central tool in understanding the principles 
governing the physical world, it plays a major role in advanced 
mathematics curricula at this level and merits significant emphasis. 
Calculus is the principal point of entry to most mathematically-based 
scientific careers. 

Students intending to study university-level mathematics also 
should develop an understanding of various components of geom- 
etry. Applications of geometry are tied directly to the solution of many 
real-world problems and are used extensively in the sciences. Since 
trigonometry has its origins in the study of triangle measurement, the 
geometry content domain also includes elements of trigonometry. 

The description of content domains for TIMSS Advanced 2008 
takes into account changes in advanced mathematics curricula 
since 1995 and, in addition, concentrates the assessment on those 
content areas for which valid and reliable measurements can be 
made with confidence. 

Even though five content domains were included in 1995, the three 
major domains were number, equations, and functions; calculus; and 
geometry, whereas probability and statistics and validation and struc- 
ture had very few items, insufficient for reporting. Items similar to 
those classified in validation and structure in 1995 will be included as 
part of the other content domains. For probability and statistics, wide 
variation across countries in both content and depth of study makes 
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it very challenging to include that content area in the TIMSS 2008 
Advanced Mathematics Study. 

In the following paragraphs, the three content domains are defined 
in terms of the specific mathematical topics to be included in the 
TIMSS 2008 Advanced Mathematics assessment. The definition of 
each content domain includes a list of broad objectives covered in the 
mathematics curriculum in the majority of participating countries. 
These objectives are presented in terms of student knowledge or the 
abilities that items aligned with the objectives are designed to elicit. 

Algebra 

Students studying pre-university mathematics should be able to make 
use of properties of the real and complex number systems to solve 
problems set in real world contexts or in abstract, mathematical ones. 
They should also have facility in investigating basic characteristics of 
sequences and series, and skill in manipulating and using combina- 
tions and permutations. The ability to work with a variety of equations 
also is fundamental for such students, providing a means of operating 
with mathematical concepts at an abstract level. The concept of func- 
tion is an important unifying idea in mathematics. 

1. Perform operations with complex numbers. 

2. Determine the n th term of numeric and algebraic series, and 
the sums to n terms or infinity of series. 

3. Solve straightforward problems involving permutations, com- 
binations, and probability. 

4. Solve linear, simultaneous, and quadratic equations and 
inequalities. Indicate whether a value (or values) satisfies a 
given equation or inequality. Solve surd (radical) equations, 
logarithmic, and exponential equations. 

5. Recognize and generate equivalent representations of func- 
tions as ordered pairs, tables, graphs, formulas, or words. 

6. Determine signs and values of functions, including rational 
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functions, for given values and ranges of the variable. Evaluate 
a function of a function. 

Calculus 

The content in this domain underpins many of the important the- 
oretical and practical applications of mathematics in a wide range 
of scientific areas and the solution of problems in many and varied 
situations. Students having taken advanced mathematics should 
demonstrate the skills, concepts, and processes acquired from the suc- 
cessful study of differential and integral calculus. 

Since the calculus content of national and local advanced math- 
ematics curricula varies considerably across countries, the calculus 
content for TIMSS Advanced Mathematics 2008 has been limited to 
material likely to be included in final year mathematics in almost all 
of the participating countries. The focus is on understanding limits 
and finding the limit of a function, differentiation and integration of 
a range of functions, and using these skills in solving problems. 

1. Evaluate limits of functions, including rational functions. 
Know the conditions for continuity and differentiability of 
functions. 

2. Differentiate polynomial, exponential, logarithmic, trigono- 
metric, rational, radical, composite, and parametric functions. 
Differentiate products and quotients. 

3. Use derivatives to solve problems (e.g., in kinematics, optimi- 
zation, and rates of change). 

4. Use first and second derivatives to determine gradient, turning 
points, and points of inflection of polynomial and rational 
functions, and sketch and interpret graphs of functions. 

5. Integrate polynomial, exponential, trigonometric, and rational 
functions. Evaluate definite integrals, and apply integration to 
compute the area under a curve. 



TIMSS Advanced 2008 Mathematics Framework: Content Domains 



Geometry 

Geometry equips students to utilize relations between points, lines, 
shapes, and properties of figures to solve problems and model physical 
situations. However, approaches to teaching content in this domain at 
this level varies across countries. Because of this, it is important to be 
clear about what strands or topics have been included in this domain 
for TIMSS Advanced 2008. 

The TIMSS geometry items relate to four strands or topics: Euclid- 
ean geometry (traditional or transformation), analytic geometry, 
trigonometry, and vectors. Euclidean geometry and analytic geom- 
etry have been important components of the secondary mathematics 
curriculum for centuries and are still widely viewed as important 
prerequisites to the study of mathematics at the university level. Trigo- 
nometry is part of the mathematics curriculum in all countries, but not 
always as part of the geometry domain. Transformational geometry 
and vectors are more recent additions to the mathematics curriculum 
in many countries, and there is considerable variation both in the 
amount of emphasis given to them across countries, as well as in the 
degree of rigor with which the area is approached. The TIMSS items 
related to these two areas deal with fairly elementary topics. 

1. Use the properties of geometric figures to solve problems. 
Prove straightforward geometric propositions in two and 
three dimensions. 

2. Use gradients, y-axis intercepts, and points of intersection of 
straight lines in the Cartesian plane in solving problems. 

3. Know and apply the equations and properties of circles in the 
Cartesian plane. Derive tangents and normals to given points 
on a circle. 

4. Use trigonometry to solve problems involving triangles. Know 
the properties of sine, cosine, and tangent graphs, and solve 
straightforward equations involving these functions. 

5. Apply the properties of vectors and their sums and differences 
to solve problems. 
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As in TIMSS 1995, students participating in the TIMSS 2008 
Advanced Mathematics assessment will be permitted to use calcula- 
tors for the entire assessment. However, it is noted that there have been 
tremendous changes in calculator technology since 1995. 
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Advanced Mathematics 
Cognitive Domains 



To respond correctly to TIMSS test items, students need to be familiar 
with the mathematics content being assessed, but they also need to 
draw on a range of cognitive skills. Describing these skills is an essen- 
tial aspect of developing the assessment of achievement in Advanced 
Mathematics because this ensures that the important cognitive goals 
of school mathematics education are surveyed across the content 
domains already defined. 

A central aim of school mathematics programs at all levels is to 
have students understand the subject matter of the courses they are 
studying. Understanding a mathematics topic consists of having the 
ability to operate successfully in three cognitive domains. The first 
domain, knowing, covers the facts, procedures, and concepts students 
need to know, while the second, applying, focuses on the ability of 
students to make use of this knowledge to select or create models and 
solve problems. The third domain, reasoning, goes beyond the solu- 
tion of routine problems to encompass the ability to use analytical 
skills, generalize, and apply mathematics to unfamiliar or complex 
contexts. 

Each content domain will include items developed to address each 
of the three cognitive domains. For example, the algebra domain will 
include knowing, applying, and reasoning items, as will the other 
content domains. 



Knowing 

Facility in using mathematics or reasoning about mathematical sit- 
uations depends on mathematical knowledge and familiarity with 
mathematical concepts. The more relevant knowledge a student is able 
to recall and the wider the range of concepts he or she has understood, 
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the greater the potential for engaging in a wide range of problem- 
solving situations and for developing mathematical understanding. 

Without access to a knowledge base that enables easy recall of 
the language and basic facts and conventions of number, symbolic 
representation, and spatial relations, students would find purpose- 
ful mathematical thinking impossible. Facts encompass the factual 
knowledge that provides the basic language of mathematics, and the 
essential mathematical facts and properties that form the foundation 
for mathematical thought. 

Procedures form a bridge between more basic knowledge and the 
use of mathematics for solving routine problems, especially those 
encountered by many people in their daily lives. In essence, a fluent 
use of procedures entails recall of sets of actions and how to carry 
them out. Students need to be efficient and accurate in using a variety 
of computational procedures and tools. They need to see that particu- 
lar procedures can be used to solve entire classes of problems, not just 
individual problems. 

Knowledge of concepts enables students to make connections 
between elements of knowledge that, at best, would otherwise be 
retained as isolated facts. It allows them to make extensions beyond 
their existing knowledge, judge the validity of mathematical statements 
and methods, and create mathematical representations. 
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Behaviors Included in the Knowing Domain 




1 



Recall definitions, terminology, notation, 
mathematical conventions, number 
properties, geometric properties. 

••••••••••••••••••••••••••••••••••••a 

Recognize entities that are 
mathematically equivalent (e.g., different 
representations of the same function or 
relation). 

••••••••••••••••••••••••••••••••••••• 

Carry out algorithmic procedures (e.g., 
determining derivatives of polynomial 

functions, solving a simple equation). 

• •••••••••••!•••••••••••••••••••••••• 

Retrieve information from graphs, tables, 
or other sources. 



Applying 

Problem solving is a central goal, and often a means, of teaching math- 
ematics, and hence this and supporting skills (e.g., select, represent, 
model) feature prominently in the domain of applying knowledge. In 
items aligned with this domain, students need to apply knowledge of 
mathematical facts, skills, procedures, and concepts to create represen- 
tations and solve problems. Representation of ideas forms the core of 
mathematical thinking and communication, and the ability to create 
equivalent representations is fundamental to success in the subject. 

Problem settings for items in the applying domain are more 
routine than those aligned with the reasoning domain and will typi- 
cally have been standard in classroom exercises designed to provide 
practice in particular methods or techniques. Some of these problems 
will have been expressed in words that set the problem situation in 
a quasi-real context. Though they range in difficulty, each of these 
types of “textbook” problems is expected to be sufficiently familiar 
to students that they will essentially involve selecting and applying 
learned procedures. 
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Problems may be set in real-life situations or may be concerned 
with purely mathematical questions involving, for example, numeric 
or algebraic expressions, functions, equations, geometric figures, or 
statistical data sets. Therefore, problem solving is included not only in 
the applying domain, with emphasis on the more familiar and routine 
tasks, but also in the reasoning domain. 



Behaviors Included in the Applying Domain 




Select 




iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiimiiiiiiiiimiiiimimiii 

Represent 




Model 
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Solve Routine 
Problems 



Select an efficient/ appropriate method 
or strategy for solving a problem where 
there is a commonly used method of 
solution. 

••••••••••••••••••••••••••••••••••••< 

Generate alternative equivalent 
representations for a given mathematical 
entity, relationship, or set of information. 

Generate an appropriate model such as 
an equation or diagram for solving a 
routine problem. 

Solve routine problems, (i.e., problems 
similar to those students are likely to 
have encountered in class). For example, 
differentiate a polynomial function, use 
geometric properties to solve problems. 
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Reasoning 

Reasoning mathematically involves the capacity for logical, system- 
atic thinking. It includes intuitive and inductive reasoning based on 
patterns and regularities that can be used to arrive at solutions to non- 
routine problems. Non-routine problems are problems that are very 
likely to be unfamiliar to students. They make cognitive demands over 
and above those needed for solution of routine problems, even when 
the knowledge and skills required for their solution have been learned. 
Non-routine problems may be purely mathematical or may have real- 
life settings. Both types of items involve transfer of knowledge and 
skills to new situations, and interactions among reasoning skills are 
usually a feature. Problems requiring reasoning may do so in differ- 
ent ways. Reasoning may be involved because of the novelty of the 
context or the complexity of the situation, or because any solution to 
the problem must involve several steps, perhaps drawing on knowl- 
edge and understanding from different areas of mathematics. 

Even though many of the behaviors listed within the reason- 
ing domain are those that may be drawn on in thinking about and 
solving novel or complex problems, each by itself represents a valuable 
outcome of mathematics education, with the potential to influence 
learners’ thinking more generally. For example, reasoning involves 
the ability to observe and make conjectures. It also involves making 
logical deductions based on specific assumptions and rules, and jus- 
tifying results. 
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Behaviors Included in the Reasoning Domain 




Analyze 



Generalize 



3 



Synthesize/ 

Integrate 



Investigate given information, and select 
the mathematical facts necessary to solve 
a particular problem. Determine and 
describe or use relationships between 
variables or objects in mathematical 
situations. Make valid inferences from 
given information. 

Extend the domain to which the result 
of mathematical thinking and problem 
solving is applicable by restating results 
in more general and more widely 
applicable terms. 

Combine (various) mathematical 
procedures to establish results, and 
combine results to produce a further 
result. Make connections between 
different elements of knowledge and 
related representations, and make 
linkages between related mathematical 
ideas. 



4 



Justify 



Provide a justification for the truth or 
falsity of a statement by reference to 
mathematical results or properties. 




Solve 

Non-routine 

Problems 



Solve problems set in mathematical 
or real-life contexts where students 
are unlikely to have encountered 
similar items, and apply mathematical 
procedures in unfamiliar or complex 
contexts. 
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TIMSS Advanced 2008 

Physics 

Framework 



PHYSICS 

FRAMEWORK 



Overview 

Similar to advanced mathematics, the physics assessment framework 
for TIMSS Advanced 2008 is organized around two dimensions: a 
content dimension specifying the domains or subject matter to be 
assessed within physics (i.e., mechanics, electricity and magnetism, 
heat and temperature, and atomic and nuclear physics) and a cog- 
nitive dimension specifying the domains or thinking processes to 
be assessed (i.e., knowing, applying, and reasoning). The cognitive 
domains describe the sets of behaviors expected of students as they 
engage with the physics content. Exhibit 2 shows the target percent- 
ages of testing time devoted to each content and cognitive domain for 
the physics assessment. 



Exhibit 2: Target Percentages for the TIMSS Advanced 2008 Physics 
Assessment Devoted to Content and Cognitive Domains 


Content Domains 


Percentages 


Mechanics 


30% 


Electricity and Magnetism 


30% 


Heat and Temperature 


20% 


Atomic and Nuclear Physics 


20% 


Cognitive Domains 


Percentages 


Knowing 


30% 


Applying 


40% 


Reasoning 


30% 
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Physics Content Domains 

The TIMSS Advanced 2008 physics assessment consists of four content 
domains: mechanics, electricity and magnetism, heat and temperature, 
and atomic and nuclear physics. The 1995 assessment also included a 
fifth content domain, wave phenomena. However, in the 2008 frame- 
work, topics from this domain were incorporated into the other four 
domains. Also, atomic and nuclear physics was called modern physics 
in 1995. 

Mechanics 

Mechanics can be regarded as the foundation of physics, since ideas 
of forces and motion are fundamental also to other areas of physics. 
Newton’s three laws of motion together with the law of gravitation 
provide the elements of this domain. Some basic features of relativity 
are included since Einstein’s theory is a significant extension of the 
classical Newtonian version of mechanics. 

1. Demonstrate basic understanding of the conditions for equi- 
librium (i.e., Newton’s first and third law, the role of torque 
and forces in equilibrium) and the dynamics of different types 
of movement (i.e., the concept of pressure in liquids). 

2. Demonstrate understanding of kinetic and potential energy 
(gravitational and elastic). Apply the notion of conservation 
of mechanical energy in relevant situations. 

3. Apply knowledge of mechanical wave phenomena in sound, 
water, and strings. Use knowledge of the relationship between 
speed, frequency, and wavelength in problem situations. Dem- 
onstrate understanding of refraction. 

4. Identify forces, including frictional force, acting on a body 
moving with constant acceleration and explain how their 
combined action influences the body’s movement. Apply 
Newton’s laws to carry out relevant calculations (e.g., of speed 
and acceleration). 
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5. Apply understanding of circular motion to find forces acting 
on a body moving in a circular path, and calculate the body’s 
centripetal acceleration, speed, and circling time. Apply the 
law of gravitation to analyze and calculate aspects of the move- 
ments of planets (e.g., distance from Sun, circling time). 

6. Demonstrates understanding of elastic and inelastic collisions. 
Apply the law of conservation of momentum in various colli- 
sion situations and the law of conservation of mechanical (i.e., 
kinetic) energy for perfectly elastic collisions. 

7. Demonstrate understanding of simple aspects of relativity 
(e.g., length contraction and time dilatation for an object 
moving with constant speed in relation to the observer). 

Electricity and Magnetism 

Phenomena related to electricity and magnetism are integral to every- 
day life. In particular, electricity is crucial for industry, business, and 
the home, providing energy in the form of heating, lighting, and power 
for a range of electric and electronic devices. The role of magnetism 
is less obvious, but through the connection between magnetism and 
electricity, magnetic phenomena are crucial for energy transforma- 
tion and transfer and our everyday electronic surroundings. The close 
relationship between electricity and magnetism is apparent in elec- 
tromagnetic radiation, with visible light an example of a particular 
interval of wave frequencies. 

1. Calculate the magnitude and direction of the electrostatic 
attraction or repulsion between isolated charged particles by 
application of Coulomb’s law. Predict the force on and the 
path of a charged particle in a homogeneous electric field. 

2. Demonstrate a thorough grasp of electrical circuits, including 
application of Ohm’s law and Joule’s law of power consump- 
tion for complex electrical circuits and for separate compo- 
nents of a circuit. 
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3. Analyze the magnitude and direction of the force on a charged 
particle in a magnetic field. Demonstrate understanding of the 
relationship between magnetism and electricity in phenom- 
ena such as magnetic fields around electric conductors, elec- 
tromagnets, and electromagnetic induction. Apply Faradays 
and Lenz’ laws of induction in problem situations. 

4. Demonstrate understanding of electromagnetic radiation 
in terms of waves caused by the interplay between varia- 
tions in electric and magnetic fields. Identify various types 
of waves (radio, infrared, x-rays, light, etc.) by wavelength 
and frequency. 

Heat and Temperature 

Heat and temperature are distinct concepts: heat is energy and as 
such can be transferred by many mechanisms, whereas temperature 
may be regarded as a measure of kinetic energy for molecules. Heat 
transfer from the Sun and between bodies of water, land mass, and 
atmosphere is the underlying cause of weather and climate on Earth. 
At varied temperatures, substances appear in the form (or phase) of 
solid, liquid, or gas. The strength and wavelengths of heat radiation is 
strongly dependent on the temperature of the radiating body. Thus, 
the color of radiating (not so for reflecting) objects can tell us about 
their surface temperature. 

1. Distinguish between heat and temperature, and identify the 
three forms of heat transfer: convection, radiation, and con- 
duction. Apply understanding of heat transfer and specific 
heat capacities to predict equilibrium temperature when 
bodies of different temperature are brought together. Apply 
knowledge of evaporation and condensation. 

2. Relate expansion of solids and liquids to temperature change. 
Apply the laws of ideal gases (in the form pV/T = constant) in 
problem situations, and understand the limitations of these laws. 
Apply the first law of thermodynamics in simple situations. 
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3. Demonstrate basic understanding of heat (“black body”) 
radiation and its dependence on temperature. Estimate the 
temperature of a body from the color of its radiation, and 
describe the fundamental principle behind the “greenhouse” 
effect on Earth. 

Atomic and Nuclear Physics 

This content domain covers much of what is sometimes known as 
modern physics, since the relevant theories and experiments were pub- 
lished during the last 100 years or so. The exploration of the atom and 
its nucleus opened a microscopic world of physics where many of the 
classical laws and concepts are no longer relevant. 

1. Describe the structure of the atom and its nucleus in terms of 
electrons, protons, and neutrons. Apply knowledge of atomic 
number and atomic mass number in problem situations. 

2. Relate light emission and absorption in the spectrum to the 
behavior of electrons. Apply understanding of the photoelec- 
tric effect in problem situations. Explain the process of x-ray 
emission by acceleration of electrons. 

3. Distinguish between types of nuclear reactions (i.e., fission, 
fusion, and radioactive decay), and discuss their role in nature 
(e.g., in stars) and society (e.g., reactors, bombs). Demonstrate 
basic understanding of radioactive isotopes, their half-lives, 
and their harmful effects on humans. 
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Physics Cognitive Domains 

To respond correctly to the TIMSS physics items, students need to be 
familiar with the physics content being assessed, but they also need to 
draw on a range of cognitive skills. Articulating these skills is crucial 
to the development of an assessment like TIMSS Advanced, since 
they are vital in ensuring that the items address the important cog- 
nitive aims of school physics education across the content domains 
already outlined. 

The cognitive dimension is divided into three domains based on 
what students have to know and do when confronting the various 
physics items developed for the TIMSS Advanced assessment. The 
first domain, knowing, covers facts, procedures, and concepts students 
need to know for a solid foundation in physics. The second domain, 
applying, focuses on the ability of the student to apply knowledge and 
conceptual understanding in a straightforward problem situation. 
The third domain, reasoning, goes beyond the solution of routine 
problems to encompass unfamiliar situations, complex contexts, and 
multi-step problems. 

While some hierarchy is imposed in the division of behav- 
iors into the three cognitive domains, a range of difficulty levels is 
expected for items developed for each of the cognitive domains. The 
following sections further describe the student skills and abilities 
defining the cognitive domains. The general descriptions are fol- 
lowed by lists of specific behaviors to be elicited by items that are 
aligned with each domain. 

Each content domain will include items developed to address each 
of the three cognitive domains. For example, the mechanics domain 
will include knowing, applying, and reasoning items, as will the other 
content domains. 
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Knowing 

Knowing refers to students’ knowledge base of physics facts, infor- 
mation, concepts, tools, and procedures. Accurate and broad-based 
factual knowledge enables students to engage successfully in the 
more complex cognitive activities essential to the scientific enter- 
prise. Students are expected to recall or recognize accurate physics 
statements; possess knowledge of vocabulary, facts, information, 
symbols, units, and procedures; and select appropriate apparatus, 
equipment, measurement devices, and experimental operations to 
use in conducting investigations. 

Behaviors Included in the Knowing Domain 

Recall Make or identify accurate statements 



Applying 

The questions addressing this cognitive domain are designed to involve 
the application of knowledge of physics facts, concepts, and proce- 
dures in straightforward problem situations. To measure applying, 
TIMSS Advanced will include physics items that require students to 
apply their understanding of physics concepts and principles to find a 
solution or develop an explanation. Items aligned with this cognitive 
domain will involve the application or demonstration of relationships, 
equations, and formulas in contexts likely to be familiar in the teaching 
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about science facts, relationships, 
processes, and concepts. Recognize 
and use scientific vocabulary, symbols, 
abbreviations, units, and scales in 
relevant contexts. 




Describe 



Describe physical materials and 
processes demonstrating knowledge 
of properties, structure, function, and 
relationships. 




and learning of physics concepts. Both quantitative problems requir- 
ing a numerical solution and qualitative problems requiring a written 
descriptive response are included. In providing explanations, students 
should be able to use diagrams or models to illustrate structures and 
relationships and demonstrate knowledge of physics concepts. 



Behaviors Included in the Applying Domain 




Relate 



imimiiiiiiiiiiiimimim 



2 



Use Models 



Relate knowledge of an underlying 
physical concept to an observed or 
inferred property, behavior, or use of 
objects or materials. 

••••••••••••••••••••••••••••••••••a** 

Use a diagram or model to demonstrate 
understanding of a physics concept, 
structure, relationship, process, or system 
(e.g., electrical circuit, atomic structure). 




4 



Find Solutions Identify or use a physical relationship, 

equation, or formula to find a qualitative 
or quantitative solution involving the 
direct application/demonstration of a 
concept. 

IIIIMIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII* •••••••••••••••••••••••••••••••••••• 

Explain Provide or identify an explanation for 

an observation or natural phenomenon 
that demonstrate understanding of the 
underlying physics concept, principle, 
law, or theory. 



Reasoning 

A major purpose of physics education is to prepare students to engage 
in scientific reasoning to solve problems, develop explanations, draw 
conclusions, and extend their knowledge to new situations. In addi- 
tion to the more direct applications of physics concepts exemplified 
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in the applying domain, problem-solving situations in the reasoning 
domain involve unfamiliar or more complicated contexts that require 
students to reason from scientific principles to provide an answer. 
Solutions may involve breaking down a problem into component 
parts, each involving the application of a physics concept or relation- 
ship. Students may be required to analyze a problem to determine 
what underlying principles are involved; devise and explain strat- 
egies for problem solving; select and apply appropriate equations, 
formulas, relationships, or analytical techniques; and evaluate their 
solutions. Correct solutions to such problems may stem from a variety 
of approaches or strategies, and developing the ability to consider 
alternative strategies is an important educational goal in the teaching 
and learning of physics. 

Students may be required to draw conclusions from physics data 
and facts, weigh advantages and disadvantages of alternative materi- 
als and processes, and evaluate solutions to problems. Considerable 
scientific reasoning also is involved in developing hypotheses and 
designing scientific investigations to test them, and in analyzing and 
interpreting data. 

Behaviors Included in the Reasoning Domain 

Analyze/ Analyze problems to determine the 

Solve Problems relevant relationships, concepts, and 
problem-solving steps. Develop and 
explain problem-solving strategies. 

imiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiimimimimimi* •••••••••••••••••••••••••••••••••••••••• 

Generalize Make general conclusions that go beyond 

the experimental or given conditions, 
apply conclusions to new situations, 
and determine general formulas for 
expressing physical relationships. 
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Synthesize/ Provide solutions to problems that 

Integrate require consideration of a number of 

different factors or related concepts. 

Make associations or connections 
between concepts in different areas of 
physics. Integrate mathematical concepts 
or procedures in the solution to physics 
problems. 

Justify Use evidence and scientific 

understanding to justify explanations 
and problem solutions. Construct 
arguments to support the reasonableness 
of solutions to problems, conclusions 
from investigations, or scientific 
explanations. 

IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII* ••••••••••••••••••••••••••••••••••••« 

Hypothesize/ Formulate hypotheses as testable 

Predict assumptions using knowledge from 

observation and/or analysis of 
scientific information and conceptual 
understanding. Make predictions 
about the effects of changes in physical 
conditions in light of evidence and 
conceptual understanding. 

Draw Detect patterns in data, describe or 

Conclusions summarize data trends, and interpolate 

or extrapolate from data or given 
information. Make valid inferences 
on the basis of evidence and/or 
understanding of physics concepts. 
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ASSESSMENT 

DESIGN 



Overview 



The TIMSS Advanced 2008 assessment measures student achievement 
in advanced mathematics and physics at the end of secondary school- 
ing. TIMSS Advanced also reports student achievement in the major 
content and cognitive domains of the two subjects. These ambitious 
coverage and reporting goals will provide valuable information for 
educators and policy makers, but pose significant challenges for data 
collection, analysis, and reporting. 

A consequence of the TIMSS Advanced reporting goals is that 
there are more items in the assessment than can be reasonably given to 
a student in the amount of testing time available. Accordingly, TIMSS 
Advanced uses a matrix-sampling approach that assembles the pool 
of achievement items in advanced mathematics and physics into a set 
of eight assessment booklets - four advanced mathematics and four 
physics booklets - with each student completing one booklet only. 
Most items appear in more than one booklet, providing a mechanism 
for linking the student responses from the various booklets. Booklets 
are distributed among sampled students so that the groups of students 
responding to each booklet are approximately equivalent in terms of 
student ability. 
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Target Population Definitions 

TIMSS Advanced plans to assess the advanced mathematics and 
physics achievement of students in the final year of secondary school- 
ing. This is the 12 th year of formal schooling in most countries. The 
target population definition for advanced mathematics is as follows: 

Students in the final year of secondary schooling 
who have taken courses in advanced mathematics. 

Similarly, for physics the target population definition is: 

Students in the final year of secondary schooling 
who have taken courses in physics. 

The decision as to which mathematics or physics courses should be 
included in defining the target population is a matter for each partici- 
pating country. In general, the courses included should be those taken 
by the most advanced students, typically those planning further study 
in mathematics or physics at university or other institutes of higher 
education. Courses should cover most of the advanced mathematics 
and physics content topics specified in this framework document. 

Depending on their course-taking experiences, students in the 
final year of schooling may belong to the advanced mathematics 
target population, the physics target population, or both. Students 
who belong to both populations will be randomly assigned either an 
advanced mathematics booklet or a physics booklet. 

Item Blocks 

To facilitate the process of creating assessment booklets, TIMSS 
Advanced assembles the assessment items into item blocks, with 
each item block consisting of approximately 10 items and requiring 
30 minutes of assessment time. TIMSS Advanced consists of 14 item 
blocks in total, seven blocks of advanced mathematics items and seven 
blocks of physics items. Student booklets are assembled from various 
combinations of these item blocks. 
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Of the 14 item blocks in the TIMSS Advanced 2008 assessment, 
six blocks consist of trend items (items that were used in the 1995 
assessment) and eight blocks consist of new items developed for the 
2008 assessment. As shown in Exhibit 3, the advanced mathematics 
blocks are labeled Ml to M7, and the physics blocks are labeled PI to 
P7. The labels shown in parentheses are the block labels used in 1995, 
where blocks I and E were 30-minute blocks and blocks } and F were 
60-minute blocks. Advanced mathematics blocks K and L from 1995, 
each containing 60 minutes of items, were released for public use when 
the results of the study were published. These are replaced in 2008 
by four 30-minute blocks, M4 through M7. Similarly, two 60-minute 
physics blocks, G and H, were released and are replaced by blocks P4 
through P7. 



Exhibit 3: 


TIMSS Advanced 2008 Block Design 




Advanced 

Mathematics Source of Items 

Blocks 




Physics 

Blocks 


Source of Items 


Ml (1) 


Block 1 from 1995 


PI (E) 


Block E from 1995 


M2 (J1) 


1 st half block J - 1995 


P2 (FI) 


I st half block F- 1995 


M3 (J2) 


2 nd half block J- 1995 


P3 (F2) 


2 nd half block F- 1995 


M4 


New items for 2008 


P4 


New items for 2008 


M5 


New items for 2008 


P5 


New items for 2008 


M6 


New items for 2008 


P6 


New items for 2008 


M7 


New items for 2008 


P7 


New items for 2008 



In 2008, the 1995 item blocks J and F each are split into two 30- 
minute blocks and relabeled M2, M3 and P2, P3, respectively. Thus, the 
2008 assessment consists exclusively of 30-minute item blocks, and all 
will be rotated. As in 1995, the 14 blocks of items contain an estimated 
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seven hours of testing time; 3 Vi hours for advanced mathematics and 
314 hours for physics. The assessment time allotted for each student 
booklet, however, is limited to 90 minutes, or three item blocks. 

Booklet Design 

The assignment of item blocks to student assessment booklets is 
designed to maximize coverage of the framework while ensuring that 
each student responds to sufficient items to provide reliable mea- 
surement of trends in advanced mathematics and physics. A further 
objective is to ensure that trends in content and cognitive domains are 
measured reliably. To enable linking among booklets while keeping 
the number of booklets to a minimum, most item blocks appear in 
two booklets. 

The 14 item blocks are distributed across eight student assessment 
booklets, as shown in Exhibit 4. 



Exhibit 4: TIMSS Advanced 2008 Booklet Design 





Booklet 


Part 1 


Assessment Blocks 
Part 2 


Part 3 




Advanced 

Mathematics 


1 


Ml 


M2 


M3 




2 


M4 


Ml 


M5 




3 


M6 


M4 


M2 




4 


M3 


M5 


M7 



Physics 


5 


PI 


P2 


P3 




6 


P4 


PI 


P5 




7 


P6 


P4 


P2 




8 


P3 


P5 


P7 
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The block-booklet assignment is identical for advanced mathemat- 
ics and physics. Each student booklet consists of three item blocks, all 
either advanced mathematics blocks (booklets 1, 2, 3, and 4) or physics 
blocks (booklets 5, 6, 7, and 8). Booklets 1 (advanced mathematics) 
and 5 (physics) consist entirely of trend blocks and are identical to 
their 1995 counterparts (booklets 3 A and 2 A respectively). All other 
booklets consist of one trend block and two new blocks. Blocks Ml 
through M5 and PI through P5 each appear in two booklets, in a 
different position in each booklet (first, second, or third). Advanced 
mathematics blocks M6 and M7 and physics blocks P6 and P7 each 
appear in one booklet only. 

Question Types and Scoring Procedures 

Student achievement in advanced mathematics and physics is assessed 
through a range of items in each subject. Two item formats are used 
in the TIMSS Advanced assessment: multiple- choice and constructed- 
response. At least half of the total number of score points come from 
multiple-choice items, each multiple-choice item being worth one 
score point. The remaining score points come from constructed- 
response items. Constructed-response items generally are worth one 
or two score points, depending on the nature of the task and the skills 
required in completing them. 

Release Policy 

Based on the TIMSS Advanced 2008 design, eight of the 14 item blocks 
are released when the assessment results for 2008 are published, and 
the remaining six blocks are kept secure for use in future assessments. 
The released blocks include four trend blocks (blocks Ml and M3 for 
advanced mathematics and blocks PI and P3 for physics) and four 
new blocks (blocks M6 and M7 for advanced mathematics and blocks 
P6 and P7 for physics). Thus, advanced mathematics blocks M2, M4, 
and M5, and physics blocks P2, P4, and P5 are retained as secure trend 
blocks for the next cycle of TIMSS Advanced. 
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Background Questionnaires 

An important purpose of TIMSS Advanced is to study the educa- 
tional context within which students learn advanced mathematics and 
physics. To collect information useful in interpreting student achieve- 
ment in participating countries, TIMSS Advanced plans a series of 
questionnaires for curriculum specialists, school principals, math- 
ematics and physics teachers, and the students themselves. 

Curriculum Questionnaires 

At earlier grades (fourth and eighth), TIMSS has successfully collected 
information about the intended curriculum using mathematics and 
science curriculum questionnaires completed by the National Research 
Coordinator, usually in concert with curriculum specialists and educa- 
tors in these subjects. The questionnaires are designed to collect basic 
information about the organization of the mathematics and science 
curriculum in each country, and about the content in these subjects 
intended to be covered up to the fourth and eighth grades. A similar 
approach is planned for TIMSS Advanced, with separate curriculum 
questionnaires for advanced mathematics and physics. These could 
provide valuable internationally-comparable information about the 
topics intended to be covered in the courses taken by students of 
advanced mathematics and physics. The curriculum questionnaires 
also could provide information on the size of the cohort of students 
that make up the advanced mathematics and physics populations in 
each country and the courses that they have taken in these subjects. 

School Questionnaire 

Completed by the principal (or designee) of each school, the school 
questionnaire asks about resources available to support advanced 
mathematics and physics instruction, school organization and instruc- 
tional time, and school environment and climate for learning. It is 
designed to take about 30 minutes. 
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Teacher Questionnaires 



The most important source of information about how the curriculum 
is taught in schools (the “implemented curriculum” in IEA parlance) 
is, of course, the classroom teacher. At fourth and eighth grades, 
TIMSS has collected very valuable data on curriculum coverage and 
instructional practices in mathematics and science through teacher 
questionnaires. Teachers of the students participating in the TIMSS 
assessments provide information about what topics their students 
have studied and the instructional approaches used. Questionnaires 
for teachers of advanced mathematics or physics have the potential 
to provide extremely useful information about classroom coverage 
of topics in these subjects, and about instructional approaches and 
practices. Whether or not it is practical to include questionnaires for 
teachers in TIMSS Advanced depends to some extent on how schools 
in participating countries are organized to teach advanced mathemat- 
ics and physics. Identifying the teachers of students of these subjects 
may be more straightforward in countries with highly-tracked edu- 
cational systems, but may pose more difficulties where students can 
choose from a wide variety of course offerings. 

Student Questionnaires 

Each student who participates in the TIMSS Advanced assessment 
will complete a questionnaire. The questionnaires for the advanced 
mathematics and physics assessments will ask about aspects of stu- 
dents’ home and school lives, including educational resources and 
home educational supports, educational experiences and aspirations, 
classroom activities related to mathematics and physics instruc- 
tion, self-perception and attitudes about mathematics and physics, 
homework and out-of-school activities, computer use, and basic 
demographic information. The student questionnaires require about 
30 minutes to complete. 
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